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Abstract 


Simulation  using  virtual  reality  (VR)  is  becoming  an  effective  tool  for  the  Army  in 
training  soldiers  to  do  their  required  tasks.  In  VR,  the  human  operator  can  interact 
with  a  wide  variety  of  computer-generated  worlds  developed  from  real  or 
imaginary  scenarios  or  both.  The  training  that  a  soldier  receives  by  simulation  is 
usually  cost  effective  to  the  Army  and  in  a  number  of  cases  is  safer  for  the 
individual  than  training  in  the  real  environment. 

Three-dimensional  (3-D)  sound  in  the  virtual  environment  (VE)  provides  a  more 
realistic  simulation  of  acoustic  environments  compared  to  diotic  (mono)  or 
dichotic  (stereo)  sound  presentation.  The  major  benefit  of  using  3-D  sound  is  that 
an  individual  can  determine  the  sound  source  direction.  When  sounds  that  are 
perceived  to  have  direction  and  sights  that  represent  virtual  objects  that  produce 
the  sounds  are  provided  through  a  head-mounted  display,  a  person  can  monitor 
and  identify  sources  of  information  from  all  possible  locations. 

The  purpose  of  this  study  was  to  determine  if  3-D  sound  generated  by  a  Tucker- 
Davis  Technologies’  3-D  sound  system  could  enhance  the  “realism”  or  fidelity  of 
the  VE.  The  main  objective  of  the  study  was  to  determine  if  an  individual  could 
distinguish  the  direction  of  a  sound  source  within  a  reasonable  degree  of  accuracy. 
Three-dimensional  sound  is  produced  by  using  a  mathematical  representation  of 
the  filtering  characteristics  of  the  pinnae  provided  through  head-related  transfer 
functions  (HRTFs).  The  HRTFs  can  be  developed  by  recording  a  generated 
broadband  sound  using  a  probe  microphone  in  the  ear  canal  and  subsequently 
dividing  the  Fourier  transform  of  the  recorded  sound  by  that  of  the  generated 
sound.  When  digital  filtering  techniques  are  used,  HRTFs  can  be  applied  to 
sounds  through  headphones.  When  an  arbitrary  sound  is  filtered  with  HRTF- 
based  filters,  the  sound  should  appear  to  come  from  specified  virtual  locations 
outside  the  earphones.  Ideally,  every  person  should  have  his  or  her  own  unique  or 
“matched”  HRTFs  to  generate  localized  sound.  Because  the  development  of 
matched  HRTFs  is  time  consuming,  generic  or  “unmatched”  HRTFs  are  used  to 
Satisfy  a  broad  range  of  listeners.  This  study  featured  a  comparison  between 
using  matched  HRTFs  versus  generic  HRTFs. 

The  results  indicate  that  the  average  localization  errors  for  the  baseline  scenario 
and  the  scenario  that  used  the  generic  HRTFs  were  small  and  close  in  value.  The 
difference,  although  statistically  significant,  has  therefore  no  practical  importance. 
The  average  localization  error  for  custom  HRTFs,  however,  was  approximately 
2.5  times  larger  than  that  of  the  baseline  scenario.  These  results  were  contrary  to 
what  should  be  expected. 
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A  COMPARISON  OF  VARIOUS  TYPES  OF  HEAD-RELATED  TRANSFER  FUNCTIONS 
FOR  3-D  SOUND  IN  THE  VIRTUAL  ENVIRONMENT 


INTRODUCTION 

Simulation  using  virtual  reality  (VR)  is  becoming  an  effective  tool  for  the  Army  in  training 
soldiers  to  do  their  required  tasks.  In  VR,  the  human  operator  is  connected  to  a  computer  that 
can  simulate  a  wide  variety  of  worlds,  both  real  and  imaginary.  The  training  that  a  soldier 
receives  by  simulation  is  usually  cost  effective  to  the  Army  and,  in  a  number  of  cases,  is  safer  for 
the  individual  than  training  in  the  real  environment.  Because  funding  and  safety  are  major  issues 
in  today’s  Army,  interests  in  VR  continue  to  grow. 

Continuous  enhancement  of  existing  virtual  environments  is  also  of  great  interest  to  both 
the  developers  and  the  users.  In  general,  as  the  virtual  environment  (VE)  becomes  more  realistic, 
training  becomes  more  effective.  That  is,  training  in  high  resolution,  high  fidelity  VEs  should 
better  prepare  soldiers  for  real-world  conditions.  Failure  to  make  such  enhancements  may  result 
in  training  that  does  not  allow  the  soldiers  to  use  their  full  capabilities  or  lead  the  users  to  believe 
that  they  have  more  capabilities  than  they  truly  have  in  a  real  environment. 

The  advancements  in  audio  displays  have  presented  an  attractive  means  of  enhancing  the 
VE.  In  any  setting,  sound  is  a  major  component  of  soldiers’  perception  of  their  environment. 

The  localization  of  sounds  guides  the  listener,  making  that  person  more  aware  of  his  or  her 
surroundings  and  increasing  his  or  her  sense  of  security.  When  quality  three-dimensional  (3-D) 
sound  is  incorporated  into  the  VE,  the  individual’s  sense  of  immersion  is  greatly  enhanced. 

BACKGROUND 

Key  steps  to  developing  a  “life-like”  simulation  have  been  initiated  through  the  “I-Port” 
program,  which  is  a  collaborative  research  and  development  program  under  the  direction  of  the 
Human  Research  and  Engineering  Directorate  (HRED)  of  the  U.S.  Army  Research  Laboratory 
(ARL).  The  I-Port  program  provides  a  reconfigurable  simulator  for  training  in  the  VE.  The 
Uniport,  a  prototype  simulator  for  the  I-Port  program,  is  a  unicycle-type  motion  platform  that 
allows  the  individual  to  move  (“pedal”)  through  the  VE.  It  extracts  the  appropriate  amount  of 
energy  from  the  individual  as  a  function  of  the  type  of  virtual  terrain  he  or  she  is  traveling.  The 
Uniport’s  successor,  the  individual  soldier  mobility  simulator  (ISMS),  provides  the  individual 
with  a  more  natural  means  of  walking  by  supporting  him  or  her  with  a  robotic  footpad  under  each 
foot.  The  footpads  passively  follow  the  movement  of  each  foot  to  provide  support  directly 
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related  to  the  virtual  terrain  on  which  the  person  is  walking.  The  addition  of  a  helmet-mounted 
display  (HMD)  or  a  rear  projection  system,  commonly  known  as  a  walk-in  simulated 
environment  (WISE),  allows  the  soldier  to  see  the  terrain  and  environment  with  which  he  or  she 
will  interact.  Finally,  real-time  simulation  of  the  environment  is  provided  through  the  Naval 
Postgraduate  School  Network  (NPSNET).  The  physical  and  visual  feedback  from  these 
simulators  provide  a  means  for  immersing  an  individual  in  a  synthetic  environment  to  some  level 
of  fidelity. 

Complete  immersion  of  an  individual  in  the  VE  is  difficult  to  accomplish.  Currently,  no 
VR  system  can  fully  facilitate  for  every  motion  that  a  human  makes  or  simulate  every  stimulus 
that  a  human  can  perceive.  However,  in  the  past  few  years,  major  advancements  have  been  made 
in  simulating  signals  that  affect  the  human  senses.  One  such  advancement  is  the  use  of  3-D 
sound  in  the  VE.  Incorporating  3-D  sound  in  the  VE  provides  a  more  realistic  simulation  of 
acoustic  environments  compared  to  a  diotic  (mono)  or  dichotic  (stereo)  sound  presentation.  The 
major  benefit  of  using  3-D  sound  is  that  an  individual  can  determine  the  sound  source  direction 
and  distance  in  space.  Individuals  hear  sounds  in  their  natural  environment  from  every  direction 
in  3-D  space,  and  the  untrained  ear  can  determine  the  sound  source’s  direction  within  a  10 
localization  error  (Wenzel,  1992).  By  generating  sounds  from  corresponding  virtual  objects 
throughout  the  VE,  a  person  has  the  potential  to  monitor  and  identify  sources  of  information 
from  all  possible  locations. 

Producing  localized  sound,  however,  is  a  complex  process.  Sounds  produced  in  the  real 
world  are  perceived  from  a  certain  location  based  on  intensity  and  time  differences  between  the 
ears  and  the  spectral  shaping  by  the  individual’s  pinnae  (outer  ear)  (Wenzel,  1992).  Specifically, 
the  outer  ear  acts  as  a  direction-dependent  sound  filter  to  aid  the  listener  in  “pinpointing”  sound 
sources.  A  mathematical  representation  of  the  filtering  characteristics  of  the  pinnae  is  provided 
by  the  head-related  transfer  function  (HRTF):  “the  complex  valued  free  field  transfer  function 
from  a  sound  source  in  a  certain  direction  to  the  eardrum.”  (Bronkehorst,  1995).  The  HRTF  can 
be  developed  by  recording  a  generated  broadband  sound  at  a  particular  spatial  location,  through  a 
probe  microphone  in  the  ear  canal  and  subsequently  dividing  the  Fourier  transform  of  the 
recorded  sound  by  that  of  the  generated  sound  (Wightman  &  Kistler,  1989).  When  digital 
filtering  techniques  are  used,  the  HRTFs  can  be  applied  to  sounds  before  those  sounds  are 
presented  through  headphones.  When  an  arbitrary  sound  is  filtered  with  HRTF-based  filters,  the 
sound  will  be  perceived  as  originating  from  specified  virtual  locations  outside  the  earphones. 
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As  with  all  human  attributes,  a  wide  degree  of  variability  exists  in  the  anatomical 
characteristics  of  the  human  head  and  ear.  Specifically,  the  pinnae  vary  in  size,  shape,  and  even 
location  on  the  human  head.  Even  minute  differences  in  these  features  can  influence  a  listener’s 
perception  of  sound  source  location.  For  the  most  accurate  spatial  representation  of  sound  in  the 
VE,  every  individual  should  use  a  unique  or  “matched”  set  of  HRTFs  to  represent  his  or  her 
perception  of  sound.  Developing  HRTFs  for  every  person  is  possible  but  is  time  consuming. 

For  this  reason,  many  commercial  applications  use  generic  or  “unmatched”  HRTFs  that  are 
sufficient  for  most  listeners  to  perceive  3-D  acoustic  images.  For  military  applications,  however, 
precision  is  important  and  generic  HRTFs  may  not  provide  enough  resolution.  Tucker-Davis 
Technologies,  Inc.  (TDT)  has  recently  developed  a  procedure  to  match  an  individual  with  a  set  of 
HRTFs  that  closely  represents  his  or  her  spatial  perceptions.  During  this  procedure,  the 
individual’s  HRTFs  are  customized  by  having  the  person  select  from  a  large  library  of  previously 
recorded  HRTFs  that  he  or  she  chose  as  the  best  representative  for  localizing  sound  at  specified 
locations. 

OBJECTIVE 

The  objectives  of  the  study  were  to  determine  (1)  if  an  individual  could  localize  a  sound 
produced  by  TDT’s  3-D  sound  system  with  reasonable  accuracy  for  use  in  a  VE,  and  (2) 
whether  individualized  HRTFs  are  necessary.  The  study  featured  a  comparison  between  using 
matched  HRTFs  versus  generic  HRTFs.  Localization  accuracy  in  the  real  environment  was  used 
as  a  baseline. 

METHODS 

Equipment 

Instrumentation  used  in  this  study  was  comprised  of  a  psycho-acoustic  signal-processing 
system,  two  computers,  and  standard  input-output  equipment.  Both  custom-made  and 
commercial  software  was  used  to  run  the  experiments.  The  specific  makes  and  models  of  the 
equipment  are  as  follow: 

a.  Tucker-Davis  Technologies  System  II  processing  system  (see  Figure  1) 

b.  International  Business  Machine  (IBM)  personal  computer  (PC)-Pentium® 

c.  Pioneer  speakers 

d.  AKG  (not  an  acronym)  acoustic  earphones  (Model  240  DF) 

e.  Polhemus  Fastrak®  tracking  system 
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Figure  1.  Tucker-Davis  Technologies  System  II  processing  system  and  Polhemus  Fastrak®, 


The  sound  stimulus  consisted  of  a  single  shot  from  an  Ml  6.  The  stimulus  was  a  good 
representative  of  broadband  sounds  that  are  easily  localized  by  listeners  with  normal  hearing  in  a 
natural  environment.  The  level  of  the  stimulus  was  adjusted  to  a  comfortable  and  consistent  level  at 
the  subject  s  ear.  The  distance  to  the  sound  source  in  both  the  real  and  virtual  environments  was 
2.5  meters. 


Customizing  Software 

Software  developed  by  TDT  provided  the  user  with  a  means  for  developing  his  or  her 
own  customized  HRTF s.  Each  individual  listened  through  earphones  to  a  generated  broadband 
sound  played  at  various  known  locations  around  the  perimeter  of  his  or  her  head.  For  each 
location,  the  listeners  selected  the  “best”  HRTF  representation.  When  they  had  completed  the 
procedure,  their  selected  HRTFs  were  used  as  part  of  the  study. 


The  software  begins  by  displaying  on  the  monitor  the  location  where  the  sound  will  be 
played.  For  example,  the  first  location  is  90°  right  of  due  forward  (at  the  right  ear).  The  user  will 
select  any  number  on  the  keypad,  and  a  previously  recorded  HRTF  assigned  to  that  number  will 
display  the  sound  at  that  location.  The  listener,  however,  will  notice  that  not  all  of  the  selections 
will  appear  to  be  coming  from  exactly  90°  right  of  due  forward.  As  mentioned  before,  differences 
in  a  person’s  features  can  influence  his  or  her  perception  of  the  sound  source  location.  When  one 
listens  to  3-D  sound  through  HRTFs  recorded  from  other  persons’  ears,  he  or  she  is  in  a  sense 
listening  through  someone  else’s  ears.  The  listeners  will  select  the  HRTF  that  they  believe  to  be 
the  best  representation  of  the  sound  coming  from  that  specific  location.  Once  the  selection  has 
been  made,  the  process  is  repeated  at  the  next  location,  120°  right  of  due  forward,  for  a  total  of 
12  locations  at  30°  increments.  The  computer  software  then  interpolates  the  HRTFs  for  the 
locations  in  between  the  30°  increments. 

Testing  Environment 

Testing  took  place  in  the  ARL  hostile  environment  simulator  (HES),  which  is  a  57-foot 
by  44-foot  by  22-foot  chamber.  A  subject  was  seated  on  a  chair  (a)  centered  within  an  array  of 
speakers  (5  meters  in  diameter)  that  were  located  every  30°  at  ear  level  (see  Figure  2)  or  (b)  with 
earphones  placed  over  the  ears  (see  Figure  3).  Background  noise  levels  in  the  chamber  were  kept 
below  recommended  noise  levels  for  audiometric  testing  using  earphone  simulation  (American 
National  Standards  Institute,  1991). 

Subjects 

Twenty  subjects  (10  males  and  10  females)  between  the  ages  of  18  and  40  were  exposed 
to  three  different  scenarios  in  which  they  tried  to  localize  sounds  from  any  direction  at  ear  level 
(0°  to  360°  azimuth).  Each  subject  had  hearing  better  than  or  equal  to  20  dB  hearing  level  (HL) 
at  audiometric  frequencies  from  250  through  8000  Hz  (American  National  Standards  Institute, 
1996),  otoscopically  normal  ears,  and  no  history  of  otologic  pathology. 

Procedure 

The  sound  stimulus  was  displayed  at  12  locations  (30°  increments)  for  each  of  the  three 
scenarios.  The  locations  of  the  sound  were  randomly  presented  to  the  listener  for  each  scenario.  In 
each  scenario,  the  subject  was  seated  in  a  revolving  chair,  wearing  either  a  headband  or  earphones 
supporting  a  Polhemus  Fastrak®  tracking  sensor  (static  accuracy  of  0.15°  root  mean  square).  The 
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subject  was  blindfolded  when  appropriate  to  prevent  visual  objects  (such  as  speakers  in  the  room) 
from  playing  a  role  in  his  or  her  response  to  the  stimulus. 
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One  scenario  served  as  a  baseline.  The  subject,  in  this  scenario,  sat  in  the  chair  and 
listened  for  sounds  produced  through  the  360°  azimuth  speaker  array.  He  or  she  was  blindfolded 
and  asked  to  determine  the  direction  of  the  produced  sound  by  turning  and  facing  the  perceived 
sound.  The  sound  was  repeated  every  3  seconds  from  the  selected  location  until  the  individual 
acknowledged  verbally  to  stop  the  sound  because  he  or  she  was  facing  the  direction  from  which 
he  or  she  perceived  the  sound  to  be  coming.  Once  the  sound  was  stopped,  the  direction  of  the 
subject  was  recorded  from  the  output  of  the  tracking  sensor. 

The  other  two  scenarios  followed  the  same  testing  procedure  as  in  the  baseline  scenario 
except  the  sound  stimulus  was  produced  through  earphones  from  the  3-D  sound  system.  The  3- 
D  sound  system  produced  the  sound  stimulus  using  generic  HRTFs  for  the  one  scenario  and 
customized  HRTFs  for  the  other  scenario.  Each  scenario  was  fully  completed  before  another 
scenario  began.  A  sample  of  the  presentation  order  for  one  subject  for  all  three  scenarios  is 
shown  in  Table  1. 


Table  1 


Sample  Configuration  for  a  Subject  to  Localize  a  Sound 
Played  at  the  Various  Locations  for  Each  Scenario 
(Twelve  locations,  every  30°,  were  randomly  selected.) 


Location  number 

1 

2 

3 

4 

5 

6 

7 

8 

9 

10 

11 

12 

Speaker  array 
(degrees) 

0 

150 

90 

180 

210 

330 

240 

120 

60 

300 

30 

270 

Generic  HRTFs 
(degrees) 

330 

210 

300 

270 

60 

120 

150 

240 

90 

180 

0 

30 

Custom  HRTFs 
(degrees) 

270 

30 

60 

240 

0 

210 

300 

120 

330 

180 

90 

150 

RESULTS  AND  DATA  ANALYSIS 

The  recorded  data  consisted  of  each  subject’s  localization  error  (in  degrees)  at  each 
location  where  the  sound  was  displayed  for  the  three  scenarios.  The  data  are  shown  in  Appendix 
A.  A  multivariate  repeated  measures  analysis  of  variance  (MANOVA)  was  conducted  on 
localization  accuracy.  The  two  main  factors  were  scenario  and  gender.  The  analysis  of  the  data 
collected  from  10  males  and  10  females  did  not  reveal  any  statistically  significant  differences 
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between  female  and  male  for  all  scenarios  (p  =  0.143).  Therefore,  all  the  data  were  combined  and 
the  conclusion  derived  from  this  analysis  applied  to  both  female  and  male  listeners.  The 
MANOVA  also  showed  that  there  was  a  statistical  difference  between  the  localization  errors  of 
each  scenario  (p  <  0.002).  Post  hoc  contrast  analysis  indicated  that  all  three  data  sets 
significantly  differed  one  from  another  (p  <  0.05) 

Table  2  provides  a  basic  comparison  between  the  results  produced  by  the  TDT  system 
(generic  and  custom  HRTFs)  and  the  real  life  sound  (baseline).  Table  2  shows  the  average 
localization  error  of  all  20  subjects  for  each  of  the  three  scenarios  determined  by  averaging  the 
subjects’  average  localization  errors  for  each  of  the  three  scenarios.  In  other  words,  each 
subject’s  localization  error  was  averaged  over  the  12  locations  measured  for  each  scenario.  Then 
the  averaged  localization  errors  for  each  subject  were  averaged  together  to  determine  the  total 
average  localization  error  for  all  subjects  in  each  scenario.  The  standard  deviations  were 
determined  in  the  same  manner. 


Table  2 


Average  Localization  Error  and  Standard  Deviation  for  Each  Scenario 


Baseline 

Generic 

Custom 

Average  (degrees) 

3.37 

4.89 

8.21 

Standard  deviation 

0.97 

1.68 

2.83 

The  results  in  Table  2  show  very  little  difference  between  the  baseline  scenario  and  the 
scenario  that  used  the  generic  HRTFs.  The  difference,  although  statistically  significant,  has 
therefore  no  practical  importance.  The  average  localization  error  for  custom  HRTFs,  however,  is 
approximately  2.5  times  larger  than  the  baseline.  These  results  are  contrary  to  what  should  be 
expected.  The  average  localization  error  for  the  custom  scenario  should  have  been  much  closer  to 
the  baseline  average  localization  error,  provided  that  the  customizing  software  was  able  to 
generate  a  compatible  match  of  each  subject’s  HRTFs.  This  indicates  that  the  customization 
procedure  developed  by  TDT  was  too  difficult  to  use  by  some  of  the  subjects. 

The  average  localization  error  for  the  custom  scenario  does  appear,  though,  to  still  be  less 
than  an  individual’s  10°  localization  error  noted  by  Wenzel.  However,  when  we  observe  the 
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individual’s  average  localization  error  for  each  scenario,  as  seen  in  Figure  4,  we  see  a  large 
variation  for  localization  error  using  the  custom  HRTFs  (s  =  2.83)  with  5  of  20  subjects  having  a 
localization  error  of  more  than  1 0°.  In  addition,  it  was  noted  that  a  number  of  individuals  stated 
that  it  was  more  difficult  to  determine  the  exact  location  of  the  sound  source  during  this  portion 
of  the  study. 


Subject 


Figure  4.  Average  localization  error  for  individuals  for  each  scenario. 


In  addition,  it  was  also  of  interest  to  look  at  the  average  localization  error  at  each  location 
for  each  scenario,  represented  in  Figure  5.  If  certain  locations  showed  a  significant  error  in 
comparison  to  the  rest  of  the  locations  (the  data  were  not  close  to  being  linear),  there  would  be  a 
need  for  further  investigation  of  whether  individuals  have  a  greater  localization  error  at  particular 
locations  from  the  initial  direction  that  they  were  facing.  Figure  5,  however,  shows  no 
discrepancies  that  would  lead  one  to  consider  that  the  precision  of  matching  was  direction 
dependent. 

CONCLUSIONS  • 

A  VR  training  simulator  must  be  versatile  and  easy  to  use.  Effective  simulator  training 
cannot  always  allow  the  individual  the  necessaiy  tune  to  be  fitted  with  some  type  of  customized 
HRTFs,  be  it  semi-custom,  as  was  done  in  this  study,  or  a  fully  measured  HRTF  that  requires 
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hours  of  recorded  sounds  through  microphones  placed  on  a  person’s  ears.  Generic  (one  size  fits 
all)  HRTFs  are  the  most  attractive  approach  if  it  provides  sufficient  accuracy  for  localization. 


Sound  Location  (degrees) 


Figure  5.  Average  perceived  sound  at  each  location  in  each  scenario. 


The  results  of  this  study  show  that  the  3-D  sound  produced  through  both  the  generic  and 
customized  FIRTFs  provided  localized  sound  with  an  average  localization  error  of  less  than  10°. 
However,  for  this  particular  study,  the  generic  HRTFs  appear  to  be  the  better  choice  for 
producing  3-D  sound.  One  reason  is  that  in  comparison  to  the  baseline,  the  average  localization 
error  recorded  using  generic  HRTFs  differed  by  less  than  2°.  Another  reason  is  that  there  was  no 
need  to  determine  an  individual’s  HRTFs  before  exposing  him  or  her  to  various  3-D  sound  cues. 

It  is  important  to  point  out  that  the  semi-custom  approach  is  an  attractive  means  for 
developing  accurate  HRTFs  for  individuals.  Currently,  though,  semi-customizing  does  require 
anywhere  from  15  minutes  to  2  hours  to  develop  a  set  of  HRTFs,  depending  on  the  thoroughness 
of  the  individual.  Being  more  thorough,  however,  did  not  necessarily  mean  better  accuracy  for 
this  study.  If  the  software  for  semi-customizing  is  improved  and  a  broader  range  of  previously 
measured  HRTFs  can  be  used,  an  individual  may  be  able  to  closely  replicate  HRTFs  similar  to 
those  he  or  she  would  obtain  if  the  HRTFs  were  measured  in  a  fraction  of  the  time. 
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LOCALIZATION  ERROR  DATA 


Females 
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1.71 

60° 

3 

6 

5 

6 

1  2 

6 

2 

5 

0 
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